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1 Android science 

Appearance and behavior 

In the evaluation of interactive robots, the performance measures are sub¬ 
jective impression of human subjects who interact with the robot and their 
unconscious reactions, such as synchronized human behaviors in the interac¬ 
tions and eye movements. 

Obviously, both the appearance and behavior of the robots are important 
factors in this evaluation. There are many technical reports that compare 
robots with different behaviors. However nobody has focused on appearance 
in the previous robotics. There many empirical discussions on very simplified 
static robots, say dolls. Designing the robot’s appearance, especially to give it 
a humanoid one, was always a role of the industrial designer. However we con¬ 
sider this to be a serious problem for developing and evaluating interactive 
robots. Appearance and behavior are tightly coupled with both each other 
and these problems, as the results of evaluation change with appearance. In 
our previous work, we developed several humanoids for communicating with 
people [3] [4] [5], as shown in Figure 1. We empirically know the effect of appear¬ 
ance is as significant as behaviors in communication. Human brain functions 
that recognize people support our empirical knowledge. 


Android Science 

To tackle the problem of appearance and behavior, two approaches are nec¬ 
essary: one from robotics and the other from cognitive science. The ap¬ 
proach from robotics tries to build very humanlike robots based on knowl¬ 
edge from cognitive science. The approach from cognitive science uses the 
robot for verifying hypotheses for understanding humans. We call this cross- 
interdisciplinary framework android science. 
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Fig. 1 . From humanoids to androids 
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Fig. 2. The framework of android science 


Previous robotics research also used knowledge of cognitive science while 
research in cognitive science utilized robots. However the contribution from 
robotics to cognitive science was not enough as robot-like robots were not 
sufficient as tools of cognitive science, because appearance and behavior can¬ 
not be separately handled. We expect this problem to be solved by using an 
android that has an identical appearance to a human. Robotics research uti¬ 
lizing hints from cognitive science also has a similar problem as it is difficult to 
clearly recognize whether the hints are given for just robot behaviors isolated 
from their appearance or for robots that have both the appearance and the 
behavior. 

In the framework of android science, androids enable us to directly ex¬ 
change knowledge between the development of androids in engineering and 
the understanding of humans in cognitive science. This conceptual paper dis¬ 
cusses the android science from both viewing points of robotics and cognitive 


science. 
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2 Development of androids 

Very humanlike appearance 

The main difference between robot-like robots and androids is appearance. 
The appearance of an android is realized by making a copy of an existing 
person. 

The thickness of the silicon skin is 5mm in our trial manufacture. The 
mechanical parts, motors and sensors are covered with polyurethane and the 
silicon skin. Figure 3 shows the silicon skin, the inside mechanisms, the head 
part and the finished product of a child android made by painting colors on 
the silicon skin. As shown in the figure, the details are recreated very well so 
they cannot be distinguished from photographs of the real child. 



Fig. 3. The silicon skin and inside mechanisms 


Mechanisms for humanlike movements and reactions 

Very humanlike movement is another important factor for developing an¬ 
droids. For realizing humanlike movement, we developed an adult android 
because the child android is too small. Figure 4 shows this developed android. 
The android has 42 air actuators for the upper torso except fingers. We de¬ 
cided the positions of the actuators by analyzing movements of a real human 
using a precise 3D motion tracker. The actuators can represent unconscious 
movements of the chest from breathing in addition to conscious large move¬ 
ments of the head and arms. Furthermore, the android has a function for 
generating facial expression that is important for interactions with humans. 
Figure 5 shows several examples of facial expression. For this purpose, the 
android uses 13 of the 42 actuators. 

The air actuator has several merits. First, it is very silent, much like a hu¬ 
man. DC servomotors that require several reduction gears make un-humanlike 
noise. Second, the reaction of the android as against external force becomes 
very natural with the air dumper. If we use DC servomotors with reduction 
gears, they need sophisticated compliance control. This is also important for 
realizing safe interactions with the android. 
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The next issue is how to control the 42 air servo actuators for realizing 
very humanlike movements. The simplest approach is to directly send angular 
information to each joint by using a simple user interface termed a motion 
editor. However we need to specify 42 angles for creating a posture, which 
takes a long time. Therefore we added a function to generate smooth motions 
based on sinusoidal signals. This is the same idea as Perlin noise [8] used 
in computer graphics. This function helps especially well in making partial 
movements; however it is still time-consuming. 



Fig. 4. Adult android developed in cooperation with Kokoro Co. Ltd. 



Fig. 5. Facial expressions of the android 


In addition to this problem, another difficulty is that the skin movement 
does not simply correspond to the joint movement. For example, the android 
has more than five actuators around the shoulder for humanlike shoulder 
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movements, with the skin moving and stretching according to the actuator 
motions. For solving this problem, a mapping table was required that corre¬ 
lates the surface movement to the actuator motions. 

Our idea for solving this problem is to train a neural network. The neu¬ 
ral network memorizes a mapping between actuator command patterns and 
marker 3D positions based on a large number of examples of android postures. 


Toward very humanlike movement 

The next step after obtaining the mapping between the surface movements 
and actuators is implementing humanlike motions in the android. A straight¬ 
forward approach for this challenge is to imitate real human motions in coop¬ 
eration with the master of the android. By attaching markers of the precise 
3D motion tracker on both the android and the master, the android can au¬ 
tomatically follow human motions. 


Humanlike perception 

The android requires humanlike perceptual abilities in addition to a human¬ 
like appearance and movements. This problem has been tackled in computer 
vision and pattern recognition in rather controlled environments. However, 
the problem becomes seriously difficult when applied to the robot in other 
situations, as vision and audition become unstable and noisy. 

Ubiquitous/distributed sensor systems solve this problem. The idea is to 
recognize the environment and human activities by using many distributed 
cameras, microphones, infrared motion sensors, floor sensors and ID tag read¬ 
ers in the environment. We have developed distributed vision systems [2] and 
distributed audition systems in our previous work. For solving this problem 
this work must be integrated and extended. 


3 Cognitive studies using androids 

Total Turing test 

As discussed in the Introduction, android science has two aspects, the en¬ 
gineering approach and the scientific approach. The most vivid experiment 
where the two approaches meet is the total Turing test. The original was de¬ 
vised to evaluate the intelligence of computers under the assumption that 
mental capacities could be abstracted from embodiment [10]. The approach 
invoked many questions about the nature of intelligence. We consider intelli¬ 
gence as subjective phenomena among humans or between humans and robots. 
Obviously, the original Turing test does not cover the concept of total intelli¬ 
gence [1]. In contrast, the android enables us to evaluate total intelligence. 
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As did the original Turing test, the Total Turing test uses a time com¬ 
petition. We have checked how many people in preliminary experiments do 
not become aware within 2 sec. that they are dealing with an android. Fig¬ 
ure 6 displays the scene. A task is given to the subject to find the colors of 
the cloth. The screen between the android and the subject opens for 2 sec. 
The subject then identifies the color. At the same time, the subject is asked 
whether he/she became aware the other is an android. We have prepared two 
types of android, one a static android and the other an android with the micro 
movements we call unconscious movements. Because a human does not freeze, 
he/she is always slightly moving even when not doing anything, such as just 
sitting on a chair. 



Fig. 6. Total Turing test 


As the result of the experiment with 20 subjects, 70% of the subjects 
did not become aware they were dealing with an android when the android 
had micro movements, but 70% became aware with the static android. This 
result shows the importance of the micro movements for the appearance of 
humanlike reality. 

The 2-second experiment does not mean the android has passed the total 
Turing test. Nevertheless, it shows significant possibilities for the android 
itself and for cross-interdisciplinary studies between engineering and cognitive 
science. 


Uncanny valley 

Why do 30% of the subjects become aware of the android? What happens if 
the time is longer than 2 sec.? In the experiment, the subjects felt a certain 
strangeness about the android’s movements and appearance. Mori [7] pre¬ 
dicted that as robots appear more human, they seem more familiar, until a 
point is reached at which subtle imperfections create a sensation of strangeness 
as shown in Figure 7. He referred to this as the uncanny valley. 
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Extension of the uncanny valley 

Why does this uncanny valley exist? We have two hypotheses: 

• If its appearance is very humanlike, the subject attempts to understand the 
android as being human. Therefore the subtle difference creates a strong 
strangeness as the uncanny valley. 

• Humans expect balance between appearance and behaviors when they 
recognize creatures. 

The second hypothesis means familiarity increases for well-balanced ap¬ 
pearance and behavior. We refer to this as the synergy effect. For example, a 
robot should have robot-like behaviors and a human should have humanlike 
behaviors [9]. This differs from the uncanny valley because humans do not 
have sensitive mental models for recognizing robots and other toys. 



Synergy effect Uncanny valley 



Based on these hypotheses, we have extended the graph depicted by Mori 
as shown in Figure 8, which was obtained by fusing the uncanny valley pro- 
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vided by the first hypothesis with the synergy effect provided by the second 
hypothesis. This 3D graph is not exact, but rather conceptual as is Mori’s 
graph. Nevertheless it is still a significant guide for our research. Our im¬ 
portant role is to verify the structure of the graph through development of 
androids and cognitive experiments with them and obtain a more precise 
graph. 

Age-dependent uncanny valley 

There is also an age-dependent relationship. One-year-old babies were at¬ 
tracted to the child android and were unperturbed by even jerky, robotic 
movements. However children between the ages of three and five were afraid 
of the android and refused to face it. We found this phenomenon with prelim¬ 
inary experiment using infants. 

We consider the reasons to be as follows. If the baby’s model of others is 
not so well-developed, the android may be able to pass itself off as human. 
Adults know the android is not human, so they do not expect it to fit closely 
a human model. However young children seem to be in the middle ground of 
applying a human model to the android, but finding it mismatches. This is 
a kind of uncanny valley. We expect to learn more about the developmental 
process of human recognition models of infants by verifying this age-dependent 
uncanny valley. 

Conscious and unconscious recognition 

Another important viewing point for the evaluation criteria is whether it is 
conscious or unconscious. The SD method evaluates conscious recognition of 
the subjects. In contrast, our previous approach evaluates the unconscious 
recognition. Which is more significant? In the evaluation of an android, this 
question is difficult to answer. In our experience, the subjects react with it as 
if it is a human even if they consciously recognize it as an android. 

We have observed the eye movement of subjects. Figure 9 shows eye move¬ 
ments between a child and the child android. The child android is very eerie 
because of the jerky movements. As shown in the figure, the subject cannot 
keep gazing on the face of the human child and often looks at the upper right 
corner. In contrast, the subject keeps gazing at the face of the android. 

Previous works in psychology suggest the following two reasons why the 
subject cannot keep gazing at the human face. 

• Arousal reduction theory: Humans shift their gazing direction to create 
barriers against external signals for concentration 

• Differential cortical activation theory: The eye movements are caused by 
brain activities. 

However these theories do not fit our experiment. We consider there is the 
third reason as follows 
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Fig. 9. Eye movements as to a human child and the android 


• Social signal theory: The eye movement is a way of representing thinking 

[ 6 ] 

We consider a human indicates he/she is social by not continually gazing 
at the face of another. 


Possibility of an android as a human 

Then, we have another experiment with the adult android that has humanlike 
behaviors. After 5 min. habituation, the subject answered questions posed by 
the android. During the habituation, the android talked while using humanlike 
body movements. Of course, the subject became aware that it was an android 
because 5 min. is enough long to observe the details. 

We have prepared two tasks for the subject. One is to respond with either 
lies or the truth to questions posed by the android. The other is to answer 
seriously both easy and difficult questions posed by the android. 

When we humans, tell a lie, it is hard to keep gazing at the face of the 
person to whom we are lying. For the first task, many subjects shift their 
gaze when they tell a lie. For the second task, almost all subjects shift their 
gaze when difficult questions are involved. With respect to the second task, 
we have compared human-human interaction and human-android interaction. 
Figure 10 shows the results that subjects shift their gaze in the same way for 
both humans and androids. 
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Fig. 10. Comparison between human-human interaction and human-android inter¬ 
action. The gazing directions are represented by 9 areas with the numbers repre¬ 
senting percentages. 
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Obviously the subjects consciously recognized the other as an android. 
However they unconsciously recognized it as a human and dealt with it as 
a social partner. Although we have discussed evaluation criteria, this finding 
suggests the evaluation process looks more complicated. 

Through the experiment, we have reached at the following hypothesis. If 
a human unconsciously recognizes the android as a human, he/she will deal 
with it as a social partner even if he/she consciously recognizes it as a robot. 
At that time, the mechanical difference is not significant; and the android can 
naturally interact and attend to human society. Verification of this hypothesis 
is not easy and will take a long time. However it is an important challenge 
that contributes to developing deeper research approaches in both robotics 
and cognitive science. 

This paper has been proposed android science as a new cross- interdisci¬ 
plinary framework. Our purpose is not to develop the androids as commercial 
products, but rather to study principles of human-robot interaction. The au¬ 
thor believes android science will contribute for it. 
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